Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression

We propose a general algorithm for approximating nonstandard Bayesian posterior distributions. The algorithm minimizes the Kullback-Leibler divergence of an approximating distribution to the intractable posterior distribution. Our method can be used to approximate any posterior distribution, provided that it is given in closed form up to the proportionality constant. The approximation can be any distribution in the exponential family or any mixture of such distributions, which means that it can be made arbitrarily precise. Several examples illustrate the speed and accuracy of our approximation method in practice

A reversible infinite HMM using normalised random measures

We present a nonparametric prior over reversible Markov chains. We use completely random measures, specifically gamma processes, to construct a countably infinite graph with weighted edges. By enforcing symmetry to make the edges undirected we define a prior over random walks on graphs that results in a reversible Markov chain. The resulting prior over infinite transition matrices is closely related to the hierarchical Dirichlet process but enforces reversibility. A reinforcement scheme has recently been proposed with similar properties, but the de Finetti measure is not well characterised. We take the alternative approach of explicitly constructing the mixing measure, which allows more straightforward and efficient inference at the cost of no longer having a closed form predictive distribution. We use our process to construct a reversible infinite HMM which we apply to two real datasets, one from epigenomics and one ion channel recording.Comment: 9 pages, 6 figure

An Empirical Study of Stochastic Variational Algorithms for the Beta Bernoulli Process

Stochastic variational inference (SVI) is emerging as the most promising candidate for scaling inference in Bayesian probabilistic models to large datasets. However, the performance of these methods has been assessed primarily in the context of Bayesian topic models, particularly latent Dirichlet allocation (LDA). Deriving several new algorithms, and using synthetic, image and genomic datasets, we investigate whether the understanding gleaned from LDA applies in the setting of sparse latent factor models, specifically beta process factor analysis (BPFA). We demonstrate that the big picture is consistent: using Gibbs sampling within SVI to maintain certain posterior dependencies is extremely effective. However, we find that different posterior dependencies are important in BPFA relative to LDA. Particularly, approximations able to model intra-local variable dependence perform best.Comment: ICML, 12 pages. Volume 37: Proceedings of The 32nd International Conference on Machine Learning, 201

Spatial Alliances of Public Transit Operators: Establishing operator preferences for area management contracts with Government

Scheduled transit services in many countries are provided by operators within geographical jurisdictions protected from competition with other public transit operators, although unprotected from competition by other modes, especially the car. This increased competition in many developed economies has led to a loss of market share of urban transit and contributed to the growing crisis in escalating costs of service provision (leading to pressure for increasing subsidy support). The response to this throughout the 1990s has seen governments progressively introducing market reforms centred on competitive tendering and economic deregulation. In more recent years, performance-based contracts have become popular variants, with an increasing number of incentive payment criteria introduced to not only promote cost efficiency but also aimed at growing patronage. Where such reform has involved area wide contracts, the boundaries of the contract areas have been essentially preserved. In recognition of the growing support for bus-based transit systems (variously referred to as bus rapid transit, busways and transitways), which offer increasing promise in growing public transit patronage, the NSW government in Australia has introduced reforms that require existing operators in the Sydney metropolitan area each currently holding an area contract (87 contracts) to work together under fifteen new spatial contracts. These new contracts overlay the existing contract areas and give incumbent operators the first option to participate. In this paper we assess ways in which operators might coalesce to deliver ongoing and new ‘regional’ services within these new trusting partnerships. Operator business preferences and potential barriers to cooperation are identified through stated preference experiments